

  * Walk-path syntax:

Walk-path,
made of various lexemes that instruct how to traverse JSON,
allows selecting a single or
multiple elements from the input JSON


There are only 2 types of lexemes (though there're a few variants of each type):
     a) Subscript lexemes - provide addressing/subscripting JSON nodes
     b) Search lexemes - perform search down the JSON tree

a. Subscript lexemes: enclosed into square braces '[', ']', their meaning depends on the content;
       following notation variants are possible:
       '[]', '[text]', '[n]', '[-n]', '[^n]', '[+n]', '[N:M]', '[N:M:S]'
       - []: an empty offset, matches an empty label, like one in the following JSON:
            { "": "empty label example" }
       - [text]: the offset selects a child in the node by the label, e.g.: '[number]' selects the
         element 3.14 in the following JSON:
            { "number": 3.14, "false": true }
       - [n]: plain numerical offset selects indexed element within an iterable (an iterable is either
         a JSON array or an object), e.g.: '[1]' - selects a 2nd child of an array (or an object) node
         (all indices and quantifiers in a walk-path are zero based); in the above JSON, the element
         the label "false" will be selected (i.e. the value 'true'')
       - [-n]: a negative numerical offset, backs off n levels up the JSON tree (from a given node);
         e.g.: a path like '[0][0][-2]' digs 2 levels down the JSON tree and then selects back the
         root node (i.e. ascends 2 levels/tiers up)
       - [^n] numerical offsets with preceding circumflex do a similar thing as negative offsets, but
         instead descends from the root; e.g.: following walk-paths results are equal: '[0][1]',
         '[0][1][2][-1]', '[0][1][2][^2]'; number n in this notation must be signless (cannot go
         negative in this type of offset) preceding '+' sign is also not allowed;
       - [+n]: a positive numerical offset makes the path iterable, e.g.: [+1] selects all immediate
         children within a given JSON node (JSON array or object) staring from the 2nd child
       - [N:M]: a range offset (same notation like in Python) - selects all elements in the specified
         range. e.g.: [-3:] selects last 3 elements among JSON iterable's children; like in Python,
         the '+','-' signs as well as the numerical indices of a range are optional, thus all of the
         following notations are valid: '[:]', '[1:]', '[-2:]', '[+2:]', '[:3]', '[:-3]', etc
       - [N:M:S]: a range offset with the step increment; S may not be zero or negative; by default
         S is set to 1; all following notations are valid: '[::]', '[::2]', '[1::2]', '[:-1:3]', etc

       Some notations may duplicate each other, e.g.:
         '[+0]' and '[:]' and '[::]', or '[+2]' and '[2:]' and '[2::1]' have the same effect

b. search lexemes: enclosed into angular braces '<', '>', instruct to perform a recursive (or a
       non-recursive) search under a given JSON tree point; following notation forms are possible:
       '<txt>', '<txt>S', '<txt>N', '<txt>SN'
       and the respective non-recursive search lexeme forms:
       '>txt<', '>txt<S', '>txt<N', '>txt<SN'
       where txt - is any text to search for, S - is an optional one letter suffix, N - is an optional
       quantifier, which comes in several variants
       - if a lexeme is given using '<..>' encasement, then a *recursive* search applied off the
         current JSON node, otherwise (i.e. '>..<' encasement given) - a *non-recursive* search is
         performed among immediate JSON node's children only
       - '<a text>': performs a search of "a text" under a JSON tree off the given node among JSON
         strings values only - it's a default behavior, which could be altered with an optional suffix
       S: an optional one letter suffix, either of these: [rRPdDNbnlLaoicewjstqQgGvkzfFuIZWS], each one
          altering the search in the following way:
         r: apply exact match (default, may be omitted) while searching JSON string values only
         R: same as r, but the lexeme expression is a Regex (regex match applied)
         P: match any string, equals '<.*>R' but faster and let saving the value into a namespace
         d: match a specific number (i.e. searches numeric JSON values only)
         D: same as d, but the lexeme expression is a Regex (the value is treated as a string here)
         N: numerical match, same as '<.*>D' but faster and let saving the value in a namespace
         b: match a boolean (i.e. searching only boolean values); if lexeme value spelled fully, i.e.:
           'true', or 'false' then matched a respective boolean, while any other (including empty)
           lexeme spelling will memorize (any) found  boolean value in the respective namespace,
           e.g.: '<true>b', '<false>b', '<>b', '<val>b'
         n: match null values only, the content within the encasement may be empty, e.g.: '><n'; if a
            lexeme is non-empty (e.g.: '<val>n), then the found null value is preserved into a
            respective namespace
         l: apply exact match while searching objects' labels only
         L: same as l, but expression in braces is a Regex (a regex match applied)
         a: match any atomic JSON value (string, numeric, boolean, null); the content within the
            encasements may be empty (e.g.: '<>a')
         o: match any object JSON value, the lexeme's content may be empty
         i: match any array (indexable) JSON value; the content within the encasement may be empty
         c: match arrays and objects (containers); the content within the encasement may be empty
         e: end-node match (matches leaves only) - matches either of: atomics, {}, []
         w: wide range match - matches any JSON value (atomics, objects, arrays)
         j: match a user specific JSON value; the content within the encasement should be a valid
            literal JSON value, e.g.: '<{ "pi": 3.14 }>j' - will find a given JSON object; it's also
            possible to specify a templated JSON, e.g.: '<{ "{lbl}": {{val}} }>j' to match JSON
            dynamically based on the current namespace values, the result of template interpolation
            still must be a valid JSON
         s: match a JSON value previously stored in a namespace, the lexeme content points to the
            namespace, e.g.: '<val>s', will find a JSON value previously stored in the 'val' namespace
         t: match a label/index value previously saved in a namespace, the lexeme must points to the
            namespace, which should be a JSON string or a numeric type only
         q: match only original JSON elements, every match is stored in the namespace, the lexeme
            may be empty
         Q: match all non-unique (duplicate) elements, every match is stored in the namespace, the
            lexeme may be empty
         g: matches JSON elements in the ascending order, the lexeme may stay empty
         G: matches JSON elements in the descending order, the lexeme may stay empty

         - Following suffixes define lexemes as directives, which do not perform any search, instead,
           directive allow performing certain actions with currently walked JSON elements:
         v: saves the currently found (or user-defined) JSON value into a namespace
         k: preserves a currently found/walked JSON element's label (index) into a given namespace;
            if the lexeme is empty, then the label (index) can be reinterpreted as a JSON value
         z: erase a namespace pointed by a lexeme's value; the lexeme cannot be empty
         f: fail-safe (or fork): if walking lexemes past the fail-safe fails, instead of progressing
            to the next iteration, a resolved JSON node (and its path) at the fail-safe point will be
            restored; when used together with 'F' directive, the walking may either continues past
            '<>F' directive, or stop at '><F' directive
         F: when spelled as '<>F' then upon walking in instructs to skip to the next iteration; if
            spelled as '><F' then instructs to stop walking (despite further path present); a non-zero
            quantifier extends behavior for both of the lexeme notations: '<>Fn' will let to continue
            walking the path at the next n'th lexeme; '><Fn' will re-iterate the walk path (up to the
            lexeme) additionally n times
         u: application-specific directive (might be even undefined), further walking is gated by the
            call-back's return status: `true` let continue walking, `false` fails walking
         I: increment (and optionally multiply) a namespace: in the notation '<val>In:m', increments
            the value 'val' by n and optionally multiplies by m (if given); the value 'val' must be
             a numeric JSON.
         Z: saves into the provided namespace a size of the currently walked JSON; recursive notation
            calculates size recursively, while non-recursive notation provides only a number of
            immediate children; with the quantifier 1 ('Z1') saves into the namespace a size of the
            currently walked JSON string (or -1 if not a string)
         W: saves into the provided namespace a current walk-path as a JSON array
         S: (re)walks preserved in a namespace path, starting from the root

       N: an integer quantifier specifying search match instance/range, comes in several variants
         n - a number (index), e.g. '<a text>3' - matches 4th encounter of a string "a text" within
           a JSON tree (off a given search point); all quantifiers, as well as numerical subscripts
           are zero based
         +n - collects all matched encounters staring from index n, e.g.: '<a text>+2' will match 3rd
           encounter of "a text", 4th, 5th and so on until all matches found
         n:m - once ':' is sighted within a quantifier, then it specifies a range; the range
           quantifiers follow the same notation/rules as subscript's range
         n:m:s - same notation rules like for [N:M:S] subscript range; if after interpolation from the
           namespace 's' happens to be negative (or zero) value, then the default value of 1 applied
       - the indices in quantifiers typically cannot go negative, with exceptions for search types
         '>..<t' and '>..<l', where they signify a relative offset, so that it's possible to select
         siblings of the found matches (see full user guide for explanation and use cases)

       All the lexemes could be bucketed by different properties:
       o  search lexemes [rRPdDNbnlLaoicewjstqQgG]
       o  directive lexemes [vkzfFuIZWS];
       o  lexemes that must not be empty [RdDLjstvzuIZWS]
       o  lexemes that optionally may stay empty [rPNbnlaoicewqQgGkfF]
       o  lexemes that may catch walked JSON values into a namespace [PNbnaoicewqQgGvkfF]
       o  search lexemes that search among labels (or indices) only [lLt]
       o  search lexemes that utilize RegExp [RDL]

All lexeme types allow specifying inner brackets, however, the closing one has to be quoted with
the preceding backslash, e.g.: '[case[0\]]' - specifies an offset lexeme to the label "case[0]";
'<tag<a\>>' - denotes a search lexeme for the text "tag<a>"

Spaces *between* lexemes are allowed and ignored; for the subscript lexemes even a more generic
rule is in play: if a subscript lexeme violates any of the described above rules and does not
represent a valid offset, then it's treated as a literal type of offset; e.g.: '[ 1]' and '[1 ]'
are in fact respective literal offsets (labels) " 1" and "1 " rather than a numerical offset 1;
'[^-3]', '[+-2]', are also examples of literal subscripts

There're cases when JSON values have to be searched where they attached only to a specific label
(a.k.a. scoped search), a following lexeme syntax facilitates such case: '[a label]:<some text>'
- in that example "some text" will be searched (and matched) only among JSON values which are
attached to the label "a label"; in that notation all suffixes (and quantifiers) are allowed
except 'l', 'L' and 't', e.g.: '[age]:<25>j:' matches all JSON numerical values 25, where it's
attached to the label "age" only



  * Usage notes:

multiple -w usage:
 - if switch -n is given, then all walk paths will be processed sequentially, otherwise, results
   would be grouped by relevance and walks interleaved; the order of provided walks will be (if
   can be) honored

options -j and -l usage:
 - when -j is given w/o -l, then each walked elements will be encapsulate into a JSON array; when
   used together, each walked elements will be grouped into relevant objects within a parent array
 - when -jj is given, walked elements with labels only (thus elements in arrays and the root are
   ignored) will be collected into JSON object; if used together with -ll then instead of
   collecting walked elements by outer label, inner labels (and respective) elements within walked
   objects will be collected

mutually exclusive primary operations:
 - options -c -i, -u, -s, -p normally are mutually exclusive; if sighted together, only one type
   will be executed (selected in the priority of the listed order); though there's a case usage
   of -p together with options -i, -u (facilitating move semantic), see notes below

options -i, -u when used with -e:
 - the options accept one parameter which may be treated differently: initially a file name
   assumed; if the file is not found, then the parameter is treated as a JSON and its parsing is
   attempted; if parsing fails then a walk-path is assumed and if it fails an exception is thrown
 - when used together with  option -e, the latter must precede option -i or option -u; in such
   case the parameter is subjected a for shell evaluation, but before that an interpolation occurs
   (see notes on the interpolation); the interpolated entry is completely escaped, thus does not
   require any additional quoting; all shell-specific chars (e.g.: '|', ';', '\"', etc) have to be
   quoted or escaped; terminate the cli line with trailing semicolon (which has to be escaped): \;

option -m usage with -i, -u:
 - option -m modifies behaviors of the these options:
   for -i: toggles "merge" semantic; by default -i can insert its parameter only into iterables,
           while with -m, the parameter is merged with the destined element(s), which could be any
           JSON type
   for -u: toggles "merge" semantic instead of overwrite when updating;
 - due to a variety of combinations of sources -i, -u and destination -w, the number of various
   operation possibilities is big, therefore it's easier to track it in the following table:

 * insert operation (-i) without merge usage examples:
   to \ from  |        [3,4]        |     {"a":3,"c":4}     |      "a":3,"c":4      |      3
 -------------+---------------------+-----------------------+-----------------------+-------------
    [1,2]     |      [1,2,[3,4]     |  [1,2,{"a":3,"c":4}]  | [1,2,{"a":3},{"c":4}] |   [1,2,3]
 {"a":1,"b":2}|    {"a":1,"b":2}    |  {"a":1,"b":2,"c":4}  |  {"a":1,"b":2,"c":4}  |{"a":1,"b":2}
     "a"      |         "a"         |          "a"          |          "a"          |     "a"
 * insert (-i) with the merge (-m):
   to \ from  |        [3,4]        |     {"a":3,"c":4}     |      "a":3,"c":4      |      3
 -------------+---------------------+-----------------------+-----------------------+-------------
    [1,2]     |      [1,2,3,4]      |       [1,2,3,4]       |       [1,2,3,4]       |   [1,2,3]
 {"a":1,"b":2}|{"a":[1,3],"b":[2,4]}|{"a":[1,3],"b":2,"c":4}|{"a":[1,3],"b":2,"c":4}|{"a":1,"b":2}
     "a"      |      ["a",3,4]      |       ["a",3,4]       |       ["a",3,4]       |   ["a",3]

 * update operation (-u) without merge usage examples:
   to \ from  |        [3,4]        |     {"a":3,"c":4}     |         "a":3         |      3
--------------+---------------------+-----------------------+-----------------------+-------------
    [1,2]     |        [3,4]        |     {"a":3,"c":4}     |           3           |      3
 {"a":1,"b":2}|        [3,4]        |     {"a":3,"c":4}     |           3           |      3
     "a"      |        [3,4]        |     {"a":3,"c":4}     |           3           |      3
 * update (-u) with the merge (-m):
   to \ from  |        [3,4]        |     {"a":3,"c":4}     |         "a":3         |      3
--------------+---------------------+-----------------------+-----------------------+-------------
    [1,2]     |        [3,4]        |         [3,4]         |         [3,2]         |    [3,2]
 {"a":1,"b":2}|    {"a":3,"b":4}    |  {"a":3,"b":2,"c":4}  |     {"a":3,"b":2}     |{"a":3,"b":2}
     "a"      |        [3,4]        |     {"a":3,"c":4}     |         {"a":3}       |      3

option -p usage with -i, -u:
  - if option -p is sighted together with -i, or -u when the latter use a walk-path as a parameter
    then the insert/update operations effectively turn into *move*: the source entries (parameters
    of the -i, -u) will be purged at the end of operation; if -pp is sighted, then everything else
    is removed besides the destination locations (-w)

options -x and -y usage:
 - these options should be given together: one -x per any number of -y options; each argument of
   -y will be prepended with preceding -x, together they will form an equivalent of -w, e.g.:
   -x1 -yA -yB -x2 -x3 -yC is converted into: -w1A -w1B -w2 -w3C; there's no syntax validation
   in -x and -y until they converted into the respective -w options, then walk validation occurs

interpolation:
 - interpolation may occur when using templates (-T), or in shell cli argument; the notation for
   an interpolation is expressed in by tokens like {name}, or {{name}}. In the latter notation
   form the token gets interpolated from the namespace pointed by the token 'name' - the JSON
   element (in the namespace) being interpolated is preserved; in the former notation format, if
   JSON is a string, then outer quotation marks are dropped, if it's a JSON array or object, then
   the respective encasements ('[', ']', or '{', '}') are dropped, so the user must specify those
 - if an empty token is given (e.g.: {}, {{}}), then the interpolation of the currently selected
   (walked) JSON element occurs (the same interpolation rules apply)

operations chaining:
 - jtc normally performs one (primary) operation at a time; if multiple required those could be
   chained up using shell piping, e.g.:
      <stream.json jtc -J | jtc <..insert operation..> | jtc <..swap operation..> | etc
   however, such approach is less preferable due to printing/parsing JSON in between each step,
   the same could be performed using option set separator '/':
      <stream.json jtc -J / <..insert operation..> / <..swap operation..> / etc
   here, all operations are performed by a single jtc invocation, resulting in a much faster
   performance, plus that way the namespace is common across all the steps, which allows achieving
   operations with JSON otherwise unattainable in the former notation

single/multiple JSONs and sources:
 - if a single JSON source is given (file, or <std-in>), then jtc will process only the first JSON
   and then quit. If the single source holds multiple JSONs and all to be processed, then option
   -a is required.
 - if multiple sources given (i.e., multiple files) then option -a is assumed and is not required,
   jtc then performs reading and parsing from multiple files concurrently (in multiple threads).
   However, if option -a is given, then it disables concurrent JSON reading/parsing and forces
   processing of all the input files sequentially (in a single thread)



  * Some examples and explanations:
  * - for a complete user guide visit:
  *   https://github.com/ldn-softdev/jtc/blob/master/User%20Guide.md

 Consider following source JSON (stored in the file example.json):
    {
        "Relation": [
            {
                "parent": "John Smith",
                "age": 31,
                "city": "New York",
                "children": [ "Sophia", "Olivia" ]
            },
            {
                "parent": "Anna Johnson",
                "age": 28,
                "city": "Chicago",
                "children": [ "John" ]
            }
        ]
    }


- to select node "children" for the first record, run:
    jtc -w[Relation][0][children] example.json
    [
       "Sophia",
       "Olivia"
    ]
  the above example shows drilling through JSON structure, but the same could be achieved via
  recursive search:
    jtc -w'<children>l' example.json


- to select all children records individually (for the first record), run:
    jtc -w[Relation][0][children][:] example.json
    "Sophia"
    "Olivia"
  or:
    jtc -w'<children>l[:]' example.json
  or:
    jtc -w'<children>l<>P:' example.json


- to select all children (for all records) and their parents as well, run:
    jtc -w[Relation][:][parent] -w[Relation][:][children][:] example.json
    "John Smith"
    "Sophia"
    "Olivia"
    "Anna Johnson"
    "John"
  or:
    jtc -w'<parent>l:' -w'<parent>l:[-1][children][:]' example.json

  the same could be achieved through a bit more succinct syntax:
    jtc -x'[Relation][:]' -y[parent] -y '[children][:]' example.json
  or:
    jtc -x'<parent>l:' -y' ' -y'[-1][children][:]' example.json

  Here, all concatenations of an option -x with every option -y is transformed into -w then
  compile and executed. That syntax is supposed to save the input space when a common path is
  present in multiple walks


- to select all children, from the node whose parent's name starts with "John", run:
    jtc -w '<^John>R[-1][children][:]' example.json
    "Sophia"
    "Olivia"

  However, the above walk syntax might be prone to false positives, as it finds any string
  starting with "John"; to improve it, we have to ensure that the search is attached to the label
  (a.k.a. scoped search), i.e., '"parent": "John..."':
    jtc -w '[parent]:<^John>R[-1][children][:]' example.json


- to add (insert) a child "James" to a parent whose name starts with "John" and reflect the
  changes right into the source file, run:
    jtc -fw'[parent]:<^John>R[-1][children]' -i'"James"' example.json
    jtc -tc example.json
    {
       "Relation": [
          {
             "age": 31,
             "children": [ "Sophia", "Olivia", "James" ],
             "city": "New York",
             "parent": "John Smith"
          },
          {
             "age": 28,
             "children": [ "John" ],
             "city": "Chicago",
             "parent": "Anna Johnson"
          }
       ]
    }


- to alter (rewrite) the parent's name from "John Smith" into "Jane Smith" run:
    jtc -fw '<John Smith>' -u '"Jane Smith"' example.json
    jtc -tc example.json
    {
       "Relation": [
          {
             "age": 31,
             "children": [ "Sophia", "Olivia", "James" ],
             "city": "New York",
             "parent": "Jane Smith"
          },
          {
             "age": 28,
             "children": [ "John" ],
             "city": "Chicago",
             "parent": "Anna Johnson"
          }
       ]
    }


- to add a new record into a JSON node where parent is '"Jane Smith"':
    jtc -tc -w'[parent]:<Jane Smith>[-1]' -i'{"gene": "Y"}' example.json
    {
       "Relation": [
          {
             "age": 31,
             "children": [ "Sophia", "Olivia", "James" ],
             "city": "New York",
             "gene": "Y",
             "parent": "Jane Smith"
          },
          {
             "age": 28,
             "children": [ "John" ],
             "city": "Chicago",
             "parent": "Anna Johnson"
          }
       ]
    }

  However, that would add a record only to "Jane Smith" rendering the whole JOSN irregular.
  It's possible to update all records conditionally: say, update all records with
  '"gene": "X"', but the records containing word "Smith" with '"gene": "Y"':
    jtc -fw'[Relation][:]<g:"X">f<\bSmith\b>R<g:"Y">v[-1]' -i0 -T'{"gene": "{g}"}' example.json
    jtc -tc example.json
    {
       "Relation": [
          {
             "age": 31,
             "children": [ "Sophia", "Olivia", "James" ],
             "city": "New York",
             "gene": "Y",
             "parent": "Jane Smith"
          },
          {
             "age": 28,
             "children": [ "John" ],
             "city": "Chicago",
             "gene": "X",
             "parent": "Anna Johnson"
          }
       ]
    }

    That conditional insertion is facilitated using a few tricks:
    1. directive `<g:"X">f` facilitates 2 things at once: it sets up a namespace `g` to the value
       "X" and provides walk branching - if subsequent walking fails, then currently walked
       point will be restored
    2. directive <g:"Y">v will rewrite namespace `g` predicated previous walk step was a success
    3. Given we would need to apply template interpolation (to utilize the namespace `g`) the
       insertion argument is irrelevant, hence '-i0' is given
    4. Finally the template `-T'{"gene": "{g}"}`` will expand into a relevant entry (before
       insertion) facilitating interpolation of a namespace `g`


- it's possible to wrap walked results back into JSON, with help of -j option:
    jtc -w'[Relation][:][parent]' -j example.json
    [
       "Jane Smith",
       "Anna Johnson"
    ]


- if we throw in an option -l, then output JSON format ensures that entries with labels will be
  grouped and displayed accordingly:
    jtc -w'[Relation][:][parent]' -jl example.json
    [
       {
          "parent": [
             "Jane Smith",
             "Anna Johnson"
          ]
       }
    ]
  or:
    jtc -w'<parent>l:' -jl example.json


- an insert and update options could be subjected for a shell cli evaluation, e.g., say we want to
  capitalize all parent names in our JSON:
    jtc -tc -w'<parent>l:' -eu echo {{}} \| tr "[:lower:]" "[:upper:]" \; example.json
    {
       "Relation": [
          {
             "age": 31,
             "children": [ "Sophia", "Olivia", "James" ],
             "city": "New York",
             "gene": "Y",
             "parent": "JANE SMITH"
          },
          {
             "age": 28,
             "children": [ "John" ],
             "city": "Chicago",
             "gene": "X",
             "parent": "ANNA JOHNSON"
          }
       ]
    }


- to add a child "Victoria" into each record, but as the first child use update operation with
  templating:
    jtc -tc -w'<children>l:' -u'<children>l:' -T'["Victoria", {}]' example.json
    {
       "Relation": [
          {
             "age": 31,
             "children": [ "Victoria", "Sophia", "Olivia", "James" ],
             "city": "New York",
             "gene": "Y",
             "parent": "Jane Smith"
          },
          {
             "age": 28,
             "children": [ "Victoria", "John" ],
             "city": "Chicago",
             "gene": "X",
             "parent": "Anna Johnson"
          }
       ]
    }
  or:
    jtc -tc -w'<children>l:' -u'["Victoria", {}];' example.json

  in the 1st example, the destination point of an update (-w), as well as the source point of the
  update (-u) are in the same input - `example.json`. In the template (-T), the interpolation
  of `{}` token will result in the naked array of "children", making resulting array carrying
  "Victoria" as the first element. -T (and its interpolation token) here refers to the -u walk;
  in the 2nd example, -u holds a template itself, which then refers to the -w walk


* for a complete user guide visit https://github.com/ldn-softdev/jtc/blob/master/User%20Guide.md

